Rethinking Safe Control in the Presence of Self-Seeking Humans

Safe control methods are often designed to behave safely even in worst-case human uncertainties. Such design can cause more aggressive behaviors that exploit its conservatism and result greater risk for everyone. However, this issue has not been systematically investigated previously. This paper uses an interaction-based payoff structure from evolutionary game theory model humans’ short-sighted, self-seeking behaviors. The captures how prior human-machine interaction experience causes behavioral strategic changes humans the long term. We then show deterministic safe techniques equilibrium-based stochastic have worse safety performance trade-offs than a basic method mediates changes. finding suggests urgent need fundamentally rethink framework used human-technology pursuit of all.